Skip to main content

DataFrame Data Structure

A DataFrame is a two-dimensional table whose rows and columns are both labeled. Each column has its own dtype, while all columns share the same row index.

import pandas as pd

people = pd.DataFrame(
    {
        "name": pd.Series(["Ada", "Lin"], dtype="string"),
        "age": pd.Series([36, pd.NA], dtype="Int64"),
    }
)

Invariants worth checking​

people.shape
people.columns
people.index
people.dtypes
people.info()

The schema is more than column names: uniqueness, dtypes, units, missingness, and index meaning all affect downstream behavior.

Columns and assignment​

names = people["name"]             # Series
subset = people[["name", "age"]] # DataFrame
people = people.assign(adult=people["age"].ge(18))

When assigning a Series, pandas aligns it by index. Assign an array-like object only when positional alignment is intentional and lengths match.

Rows and mutation​

Select and assign in one explicit operation:

people.loc[people["name"].eq("Lin"), "age"] = 34

Avoid chained assignment such as df[mask]["age"] = 34. It obscures which object is modified and conflicts with pandas' copy-on-write model.

Prefer transformations that return a result (assign, rename, drop) over scattered inplace=True mutations. Rebinding makes pipeline stages easier to inspect and test.

Boundary​

Pandas works best for tabular data that fits comfortably in memory. For strict schemas, enforce them at ingestion; for larger-than-memory or distributed work, choose an engine designed for that execution model.

Source​